Combining Content-Based and URL-Based Heuristics to Harvest Aligned Bitexts from Multilingual Sites with Bitextor

Nowadays, many websites in the Internet are multilingual and may be considered sources of parallel corpora. In this paper we will describe the free/open-source tool Bitextor, created to harvest aligned bitexts from these multilingual websites, which may be used to train corpusbased machine translation systems. This tool uses the work developed in previous approaches withmodifications and improvements in order to obtain a tool as adaptable as possible tomake it easier to process any kind of websites and work with any pairs of languages. Content-based andURL-based heuristics and algorithms applied to identify and align the parallelwebpages in awebsite will be described and, finally, some results will be presented to show the functionality of the application and set the future work lines for this project.